1. Field of the Invention
The present invention relates to a control system for controlling the direction of travel of a vehicle, and in particular to a control system having an embedded spatial database using optical tracking for vehicle control and guidance. The control system of the present invention may also be used to control other aspects of a vehicle's motion, such as speed or acceleration. Furthermore, in the case of agricultural vehicles and the like, the present control system may be used to control yet other aspects of the vehicle's operation, such as the application of agricultural chemicals at desired locations (including at desired application rates), or the engagement and/or mode of operation of agricultural implements (e.g., plows, harvesters, etc.) at desired locations, etc.
For convenience, the invention will be described mainly with reference to agricultural vehicles and moving agricultural machinery. However, it will be clearly understood that the invention is not limited to agricultural applications and it may equally be applied to vehicles and other moving machinery in other areas.
2. Description of the Related Art
Automatic control of steering (“autosteering”) of vehicles is becoming more widespread, especially in agricultural and mining applications. Most commercially available automatic steering systems include a controller that has means for determining, among other things, the position and heading of a vehicle, a computer-based system for comparing the position and heading of the vehicle with a desired position and heading, and a steering control responsive to a control signal issued by the controller when the position and/or heading of the vehicle deviates from the desired position and/or heading.
A number of control systems have previously been devised for controlling the steering of agricultural vehicles. These systems are generally used on vehicles such as tractors (including tractors with towed tools or other implements), harvesters, headers and the like which operate in large fields. These vehicles generally move along predetermined trajectories (“paths”) throughout the field. In general, a wayline is entered into the control system and subsequent paths are calculated based on the wayline. If the vehicle deviates from the path as it moves, the controller causes the vehicle to steer back towards and onto the path as described below.
As the vehicle moves along the predetermined path trajectory, it uses various means such as signals produced by GPS (global positioning system) or INS (inertial navigation system) to identify if the vehicle deviates from the desired path trajectory. If the vehicle deviates, the extent of the deviation (i.e. the difference between the actual curvature of the vehicle's trajectory and the desired curvature, its actual compass heading compared with the desired compass heading, and the distance the vehicle is displaced laterally from the desired path) is expressed in the form of an error, and this error is fed back into the control system and used to steer the vehicle back onto the desired path.
A problem with previous vehicle control systems is that they are inherently “one-dimensional” or “linear” in nature. This means that, at a fundamental level, the controller operates by “knowing” the path that the vehicle is required to traverse, and “knowing” where the vehicle is located on that path (i.e. how far along the path the vehicle has moved) at a given time. However, the controller does not “know” where the vehicle is actually located in space. This is despite the fact that the controller may often progressively receive information containing the vehicle's spatial location, for example from the GPS/INS signals. In current controllers, the GPS/INS signals are used primarily to determine when the vehicle deviates from the path (i.e. to calculate the error) rather than for the primary purpose of determining the vehicle's actual position in space. Hence, at a fundamental level, the controller only “knows” the geometry of the path and how far the vehicle has moved along the path.
Therefore, with current controllers, if it is desired to know the actual spatial position of the vehicle, this must be calculated from the known geometry of the path and the known distance the vehicle has moved along that path. This calculation can be computationally expensive and difficult to implement in practice, particularly for curved, piecewise, broken or other complex path trajectories.
By way of example, it will be appreciated that one form of common path trajectory that agricultural vehicles are often required to traverse in fields is made up of a number of (usually parallel) path segments or “swaths” (these are sometimes also referred to as “rows”). Thus, the vehicle typically moves along one swath, harvesting or plowing as it goes, and it then turns around and moves back along an adjacent parallel swath, harvesting or plowing in the opposite direction. The adjacent swath will generally be spaced from the first swath sufficiently closely that no part of the field or crop is missed between the swaths, but also sufficiently apart so that there is not an unnecessary overlap region (i.e. a region between the swaths that gets plowed or harvested on both passes). In general, the distance between the mid-lines of each respective swath is determined with reference to the width of the vehicle (i.e. the width of the plow, harvester or possibly the tool being towed by the vehicle).
In cases where paths comprising a series of parallel swaths are used, the first swath will often be used as a reference swath or “wayline”. In general, the geometry of the wayline in space will be entered into the control system along with the vehicle or implement width, and this is used to calculate the required spacing (and hence trajectory) for each of the adjacent parallel swaths. However, with most existing control systems, the controller is only able to control the steering of the vehicle as it proceeds along each of the swaths. It is much harder to control the steering of the vehicle as it turns around between one swath and the next. Therefore, whilst the spatial geometry of the respective swaths may have been calculated, from the control system's point of view at any given time it only “knows” that it is on the nth swath (numbered from the wayline) and that it has been moving along that swath for a known amount of time with known speed (i.e. it knows that the vehicle is a certain distance along the nth swath). However, at a fundamental level, the control system does not inherently know where the vehicle is consequently located in space or the spatial relationship between each swath. A graphical representation of the difference between the vehicle's actual spatial location and what the control system “sees” is given in FIG. 1.
The “one-dimensional” or “linear” nature of existing control systems also causes other difficulties. One example is in relation to obstacle avoidance. In most agricultural applications, the positions of obstacles (e.g., fences, trees, immovable rocks, creeks, etc.) are known according to their “real-world” spatial location. The spatial location may be known according to global latitude and longitude coordinates (e.g., as provided by GPS), or alternatively the location may be known relative to a fixed point of known location (this is generally a point in or near the field used to define the origin of a coordinate system for the field). However, as current control systems only recognize where the vehicle is located along the path, not where the vehicle is actually located in space, the control system itself is therefore unable to recognize whether the location of the obstacle coincides with the trajectory of the path, and hence whether there may be a collision.
Consequently, with current control systems, it may be necessary for a number of separate modules to be provided, in addition to the primary control module, if automatic obstacle avoidance (i.e. obstacle avoidance without the need for intervention by the driver of the vehicle) is to be achieved. In these cases, one of the modules would be a collision detection module for calculating the geometry and trajectory of a section of the path a short distance ahead of the vehicle in terms of “real world” spatial coordinates and for determining whether any of the points along that section of path will coincide with the location of an obstacle. If the collision detection module identifies that the section of path is likely to pass through an obstacle (meaning that there would be a collision if the vehicle continued along that path), then a further module may be required to determine an alternative trajectory for (at least) the section of the path proximate the obstacle. Yet a further module may then be required to determine how best to steer the vehicle from the alternative trajectory back onto the original path after the vehicle has moved past the obstacle. This multi-modular control system structure is complicated and can lead to computational inefficiencies because the different modules may each perform many of the same geometric calculations for their own respective purposes, separately from one another, leading to “doubling up” and unnecessary computation. Also, with this modular control system structure, control of the vehicle generally passes from one module to another as described above, but determining when one module should take over from another creates significant difficulties in terms of both system implementation and maintenance.
Another problem associated with the “one-dimensional” nature of existing control systems is their inherent inflexibility and unadaptability. For example, in practice, if the vehicle deviates from the desired path for some reason, it may be preferable for subsequent paths (swaths) to also include a similarly shaped deviation so that the paths remain substantially parallel along their length (or tangentially parallel and consistently spaced in the case of curved sections of path). If the vehicle is, for example, a harvester or a plow, then keeping the paths parallel in this way may help to prevent portions of the field from being missed, or from being harvested/plowed multiple times (by passing over the same portion of field on multiple passes). Even with the modular control system structures described above, it is often difficult to determine the geometry of the deviated path portion in terms of “real world” coordinates, and even if this can be done, it is also difficult to adjust subsequent path geometries to correspond to the deviation from the predetermined path trajectory that was originally entered.
As a further example of the inherent inflexibility and unadaptability of current “one-dimensional” control systems, it is illustrative to consider the situation where an obstacle is located near the end of one swath such that it would be quicker and more efficient to simply move on to an adjacent swath located nearby rather than wasting time trying to go around the obstacle to finish the first swath before moving on to the adjacent swath. Current “one-dimensional” control systems are not able to recognize that it would be more efficient to move on. This is because the control system only knows where the vehicle is along its current path (e.g., close to the end of the swath), and if a modular control systems is used, that module may also recognize that it is approaching the obstacle. The control system does not know where the vehicle is actually located in space, and therefore it cannot recognize that the beginning of the next swath is actually located nearby—it simply does not know where the next swath is (or indeed where the current swath is in space). Therefore, current control systems cannot easily recognize when it would be better to change paths (at least without intervention from the vehicle's driver), as this example illustrates. Nor is the current “one-dimensional” structure inherently adapted to enable the control systems to automatically (i.e. autonomously without assistance from the driver) determine and guide the vehicle along an efficient trajectory between swaths.
As used herein, “attitude” generally refers to the heading or orientation (pitch with respect to the Y axis, roll with respect to the X axis, and yaw with respect to the Z axis) of the vehicle, or of an implement associated with the vehicle. Other vehicle/implement-related parameters of interest include groundspeed or velocity and position. Position can be defined absolutely in relation to a geo-reference system, or relatively in relation to a fixed position at a known location, such as a base station. A change in one or both of the position and orientation of the vehicle (which can include a towed component, such as an implement or a trailer) can be considered a change in the vehicle's “pose.” This includes changes (e.g., different order time derivatives) in attitude and/or position. Attitude and position are generally measured relatively with respect to a particular reference frame that is fixed relative to the area that the vehicle is operating in, or globally with respect to a geo-reference system.
U.S. Pat. No. 6,876,920, which is assigned to a common assignee herewith and incorporated herein by reference, describes a vehicle guidance apparatus for guiding a vehicle over a paddock or field along a number of paths, the paths being offset from each other by a predetermined distance. The vehicle guidance apparatus includes a GNSS receiver for periodically receiving data regarding the vehicle's location, and an inertial relative location determining means for generating relative location data along a current path during time periods between receipt of vehicle position data from the GNSS receiver. The apparatus also includes data entry means to enable the entry by an operator of an initial path and a desired offset distance between the paths. Processing means are arranged to generate a continuous guidance signal indicative of errors in the attitude and position of the vehicle relative to one of the paths, the attitude and position being determined by combining corrected GNSS vehicle location data with the relative location data from the inertial relative location determining means.
In the system described in U.S. Pat. No. 6,876,920, the inertial sensor is used to provide a higher data rate than that obtainable from GNSS alone. Although the inertial navigation system (INS) part of the steering control system suffers from errors, in particular a yaw bias, the signals received from the GNSS system are used to correct these errors. Thus, the combination of a GNSS based system and a relatively inexpensive INS navigation system allow for quite accurate control of the position of the vehicle. Although this system allows for accurate vehicle positioning and sound control of the vehicle's steering, difficulties may be experienced if there are prolonged periods of GNSS outage. GNSS outages may occur due to unsuitable weather conditions, the vehicle operating in an area where GNSS signals cannot be accessed, or due to problems with the GNSS receiver. If a period of prolonged GNSS outage occurs, the steering system relies solely upon the INS. Unfortunately, a yaw bias in a relatively inexpensive inertial sensor used in the commercial embodiment of that steering control system can result in errors being introduced into the steering of the vehicle.
Optical computer mice are widely used to control the position of a cursor on a computer screen. Optical computer mice incorporate an optoelectronic sensor that takes successive pictures of the surface on which the mouse operates. Most optical computer mice use a light source to illuminate the surface that is being tracked (i.e. the surface over which the mouse is moving). Changes between one frame and the next are processed using the image processing ability of the chip that is embedded in the mouse. A digital correlation algorithm is used so that the movement of the mouse is translated into corresponding movement of the mouse cursor on the computer screen.
The optical movement sensors used in optical computer mice have high processing capabilities. A number of commercially available optical computer mice include optical mouse sensors that can process successive images of the surface over which the mouse is moving at speeds in excess of 1500 frames per second. The mouse has a small light emitting source that bounces light off the surface and onto a complementary metal oxide semiconductor (CMOS) sensor. The CMOS sensor sends each image to a digital signal processor (DSP) for analysis. The DSP is able to detect patterns in images and see how those patterns have moved since the previous image. Based on the change in patterns over a sequence of images, the digital signal processor determines how far the mouse has moved in X and Y directions, and sends these corresponding distances to the computer. The computer moves the cursor on the screen based upon the coordinates received from the mouse. This happens hundreds to thousands of times each second, making the cursor appear to move very smoothly.
The chips incorporated into optical computer mice often include photodetectors and an embedded integrated circuit that is used to analyse the digital signals received from the photodetectors. The photodetectors may include an array of photosensors, such as an array of charged couple devices (CCDs).
U.S. Pat. No. 5,786,804 (incorporated herein by reference), which is assigned to Hewlett-Packard Company, describes a method and system for tracking attitude of a device. The system includes fixing a two-dimensional (2D) array of photosensors to the device and using the array to form a reference frame and a sample frame of images. The fields of view of the sample and reference frames largely overlap, so that there are common image features from frame to frame. Several frames are correlated with the reference frame to detect differences in location of the common features. Based upon detection of correlations of features, an attitudinal signal indicative of pitch, yaw and/or roll is generated. The attitudinal signal is used to manipulate a screen cursor of a display system, such as a remote interactive video system.
It will be clearly appreciated that any reference herein to background material or a prior publication is not to be understood as an admission that any background material, prior publication or combination thereof forms part of the common general knowledge in the field, or is otherwise admissible prior art, whether in Australia or any other country.